Skip to main content

​
Overview

enrich_fno_data.py is a Phase 4c pipeline script that enriches the master stock database with Futures & Options (F&O) data. It fetches current lot sizes and expiry calendar information from Dhan’s API and maps them to stocks flagged as F&O-eligible in the master ISIN map.
Pipeline Position: Phase 4c - Final enrichment step before data exportCritical Function: Adds trading-essential F&O metadata required for options and futures analysis

​
Purpose

This script:
  • Identifies F&O-eligible stocks from master_isin_map.json
  • Fetches real-time lot sizes from Dhan
  • Retrieves next expiry dates from the F&O calendar
  • Enriches all_stocks_fundamental_analysis.json with F&O metadata

​
Input Files

​
all_stocks_fundamental_analysis.json
JSON
required
Master stock database containing all fundamental and technical data
​
master_isin_map.json
JSON
required
ISIN mapping file with FnoFlag field (1 = F&O eligible, 0 = not eligible)

​
Output Files

​
all_stocks_fundamental_analysis.json
JSON
Updated master file with three new fields added to each stock record:
  • F&O: “Yes” or “No”
  • Lot Size: Integer lot size or “N/A”
  • Next Expiry: Date string (YYYY-MM-DD) or “N/A”

​
Processing Logic

​
1. Build ID Extraction

Dynamically fetches the Next.js buildId from Dhan’s lot size page: 
def get_build_id():
    """Dynamically fetch the Next.js buildId from a Dhan page."""
    url = "https://dhan.co/nse-fno-lot-size/"
    try:
        response = requests.get(url, headers={"User-Agent": get_headers()["User-Agent"]}, timeout=10)
        match = re.search(r'"buildId":"([^"]+)"', response.text)
        return match.group(1) if match else None
    except:
        return None

​
2. Lot Size Fetching

Retrieves current month lot sizes for all F&O instruments: 
def fetch_lot_sizes(build_id):
    """Fetch current F&O lot sizes. Returns {symbol: lot_size_current_month}."""
    lot_map = {}
    if not build_id:
        return lot_map

    url = f"https://dhan.co/_next/data/{build_id}/nse-fno-lot-size.json"
    headers = {"User-Agent": get_headers()["User-Agent"]}

    try:
        r = requests.get(url, headers=headers, timeout=15)
        if r.status_code == 200:
            data = r.json()
            instruments = data.get("pageProps", {}).get("listData", [])
            for item in instruments:
                sym = item.get("sym")
                fo_contracts = item.get("fo_dt", [])
                if sym and fo_contracts:
                    # First contract = current month lot size
                    lot_map[sym] = fo_contracts[0].get("ls")
    except:
        pass

    return lot_map

​
3. Expiry Date Fetching

Retrieves the next upcoming expiry date for each F&O symbol: 
def fetch_next_expiry(build_id):
    """Fetch F&O expiry calendar. Returns {symbol: next_expiry_date}."""
    expiry_map = {}
    if not build_id:
        return expiry_map

    url = f"https://dhan.co/_next/data/{build_id}/fno-expiry-calendar.json"
    headers = {"User-Agent": get_headers()["User-Agent"]}

    try:
        r = requests.get(url, headers=headers, timeout=15)
        if r.status_code == 200:
            data = r.json()
            expiry_raw = data.get("pageProps", {}).get("expiryData", {}).get("data", [])

            from datetime import datetime
            today = datetime.now().strftime("%Y-%m-%d")

            for exchange_data in expiry_raw:
                for exp_group in exchange_data.get("exps", []):
                    for item in exp_group.get("explst", []):
                        sym = item.get("symbolName")
                        exp_date = item.get("expdate")
                        if sym and exp_date and exp_date >= today:
                            # Keep the nearest future expiry
                            if sym not in expiry_map or exp_date < expiry_map[sym]:
                                expiry_map[sym] = exp_date
    except:
        pass

    return expiry_map

​
4. Data Enrichment

Maps F&O data to stocks based on the FnoFlag: 
# Enrich master JSON
enriched = 0
for stock in master_data:
    sym = stock.get("Symbol")

    if sym in fno_symbols:
        stock["F&O"] = "Yes"
        stock["Lot Size"] = lot_map.get(sym, "N/A")
        stock["Next Expiry"] = expiry_map.get(sym, "N/A")
        enriched += 1
    else:
        stock["F&O"] = "No"
        stock["Lot Size"] = "N/A"
        stock["Next Expiry"] = "N/A"

​
Fields Added

​
F&O
string
Indicates F&O eligibility
  • "Yes" - Stock is F&O eligible
  • "No" - Stock is not F&O eligible
​
Lot Size
string | number
Current month lot size for F&O contracts
  • Integer value (e.g., 500, 1000) for F&O stocks
  • "N/A" for non-F&O stocks
​
Next Expiry
string
Next upcoming expiry date
  • Date in YYYY-MM-DD format (e.g., "2026-03-26") for F&O stocks
  • "N/A" for non-F&O stocks

​
Usage Example

python enrich_fno_data.py
Expected Output: 
Found 243 F&O eligible stocks from ISIN map.
Fetching Dhan buildId...
  BuildId: abc123xyz
Fetching F&O lot sizes...
  Got lot sizes for 243 instruments.
Fetching F&O expiry calendar...
  Got expiry dates for 243 instruments.
Successfully enriched 243 F&O stocks in master JSON.

​
Error Handling

  • Missing master_isin_map.json: Script continues but no stocks are marked as F&O eligible
  • Build ID fetch failure: Returns empty lot size and expiry maps
  • API timeout: Individual fetch functions fail gracefully, returning empty dictionaries
  • Symbol mismatch: Stocks not found in Dhan data receive “N/A” values

​
Data Source

  • F&O Eligibility: master_isin_map.json (FnoFlag field)
  • Lot Sizes: Dhan NSE F&O Lot Size API
  • Expiry Dates: Dhan F&O Expiry Calendar API